The stereo image pair is used to record the same scene shot by video cameras at different positions. Image points of the same object point on two different imaging planes are called corresponding points, and range information about the scene can be obtained from disparity of the corresponding points in the stereo image pair. As the stereo image pair provides epipolar geometric constraints, the corresponding points can be obtained from the corresponding epipolar lines. Generally, two video cameras are required in a stereo vision system. FIG. 1 shows a model of a stereo vision system including two video cameras, where (o, o′) denotes the optical centers of the video cameras, point M is a point in the space, and m and m′ respectively denote the images of the space point M shot by the two video cameras from two opposite perspectives.
In the coordinate system of the video camera, the epipolar line constraints exist: The line oo′ intersects the image planes R and R′ at points e and e′, which are called epipoles. Lines through the epipoles on the two images are called epipolar lines. For any point on the first image plane, its corresponding point on the other image plane is bound to locate on its epipolar line. As shown in FIG. 1, for point m, line l that is through point m and point e is an epipolar line of the first image plane; point m′ is the corresponding point of point m and locates on line l′ that is through point e′. That is, mapping between point m′ and epipolar line l′ of the second image plane exists.
For an ideal stereo image pair, only horizontal disparity exists between the corresponding points. To match the ideal stereo image pair, it is only necessary to identify the corresponding points along the horizontal scanning line of the image. Therefore, the ideal stereo image pairs can be matched at a high speed, and ambiguity of matching can be eliminated. Thus, the process of rectifying the stereo image pair is a process of transforming a general stereo image pair into an ideal stereo image pair.
A stereo image pair can be rectified in two scenarios: one refers to a video camera that is calibrated, and the other refers to a video camera that is not calibrated.
In the case of a video camera that is calibrated, the stereo image pair is rectified after the internal and external parameters of the video camera are obtained. The rectification process includes: projecting the original images onto a public image plane, where the direction of this image plane is determined by the cross product by the baseline of the binocular stereo vision system and the intersecting line of two original image planes.
In the case of a video camera that is not calibrated and a series of corresponding points are obtained, before the stereo image pair is rectified, angular point sets need to be obtained from the image pair through the angular point extraction algorithm respectively, mapping between angular points in the two angular point sets needs to be established through the feature matching method, and then point pair sets are established, in which two corresponding angular points form a point pair.
The inventor finds the following problems in the prior art.
(1) In rectifying the stereo image pair, epipolar lines corresponding to the stereo image pair can be made parallel to each other only, but not located on the same horizontal line.
(2) The conventional algorithm for rectifying the stereo image pair is of poor adaptability, and can achieve good effect only when the binocular video camera platform is constructed well. When the binocular video camera platform is distinctly inferior to an ideal binocular video camera or is not well constructed due to mechanical or manual factors, rectification of the stereo image pair is not good.
(3) Application to the stereo video is not taken into account. Generally, most of the rectified images may generate black edges, but the method for favorably trimming the black edges is not taken into account.